Unbalanced edge modes and topological phase transition in gated trilayer graphene 
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Gapless edge modes hosted by chirally-stacked trilayer graphene display unique features when a 
bulk gap is opened by applying an interlayer potential difference. We show that trilayer graphene 
with half-integer valley Hall conductivity leads to unbalanced edge modes at opposite zigzag bound- 
aries, resulting in a natural valley current polarizer. This unusual characteristic is preserved in the 
presence of Rashba spin-orbit coupling that turns a gated trilayer graphene into a Z-z topological 
insulator with an odd number of helical edge mode pairs. 
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Introduction — Gapless edge modes in two-dimensional 
condensed matter physics often appear at surfaces or 
boundaries of materials with a well-defined topological 
order in the bulk. Well-known examples include the chi- 
ral quantum Hall edge states characterized by the first 
Chcrn number CX, spin-helical edge modes in the Z2 topo- 
logical insulators protected by time reversal invariance 2 , 
valley- helical edge modes in graphene 3 , and kink states 
arising between regions of inverted bulk orbital moments 
or valley-Hall conductivities^—. Recently, multilayer 
graphene with chiral stacking has generated lots of in- 
terest in the community 11 . Such a system can develop a 
valley-Hall conductivity in the presence of a gap opening 
mass term 7 , including gapped bilayer 5 and single layer 6 . 
For all these known examples so far it is expected that 
the edge modes (chiral or helical) are distributed in equal 
numbers at opposite boundaries. 

In this Rapid Communication, we study the special be- 
havior of edge modes in chirally-stacked trilayer graphene 
in the presence of an interlayer potential difference. We 
show that gated trilayer graphene has an unusual spa- 
tially unbalanced distribution of valley-Hall edge modes 
at sample boundaries with unequal pairs of edge modes, 
resulting in a natural valley current polarizer. We also 
show that the gated trilayer can be brought into a topo- 
logical insulator phase by introducing the Rashba spin- 
orbit coupling (SOC). In this case the edge states are also 
distributed unevenly and the system is characterized by 
different odd numbers of edge modes pairs located at op- 
posite sample boundaries. 

System Hamiltonian — In the following, we present the 
general form of the tight-binding Hamiltonian of a gated 
trilayer graphene in the presence of Rashba SOC 

H = Hj LG + Hli G + Hi LG + t ± J2 °h 
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where -ff s £ G ' represent respectively the monolayer 
graphene Hamiltonian of the top (T), middle (M) and 



bottom (B) layers, and can be written as 
Hslg = -tj^cjcj+itn, Yl ( Sc ^ x d v)* c h c J0' ( 2 ) 

(ij) (ij)a@ 

where c\ creates an electron on site i, and t is the 
intralayer hopping energy between nearest neighbours. 
The Rashba SOCi 2 - strength is measured by £r, where s 
are spin Pauli matrices, and d,j describes a lattice vec- 
tor pointing from site j to site i. The interlayer hopping 
tx couples two neighbouring layers in a Bernal stacking 
pattern. Finally, the gate bias 2U is applied by setting 
the lattice site potentials to be +U, 0, and —U on the 
top, middle, and bottom layers, respectively. 

Quantum valley-Hall edge modes — When a perpendic- 
ular electric field is applied on a multilayer graphene, a 
nontrivial bulk gap opens to host a quantum valley-Hall 
state. This is characterized by a valley Hall conductivity 
given by a v xy — (a^ y — a^ y )e 2 /2h for the spinless case^ 7 ^ 3 -, 
where c;^ K is obtained by using a continuum model at 
K or K'. When these two valleys are separated and in- 
tervalley scattering is avoided, such valley Hall conduc- 
tivity assumes an integer (semi-integer) value for even 
(odd)-TV layers of graphene stacks^. Such a bulk quan- 
tization only has edge correspondence at specific system 
boundaries. For example, zigzag ribbon geometries with 
large momentum separation between valleys^ can sup- 
port valley-Hall edge modes, manifesting the quantized 
valley-Hall conductivity of the bulk 7 . For even N, we 
can see an integer number of valley-Hall edge mode pairs 
located at both edges in keeping with the integer quanti- 
zation of valley-Hall conductivity. For odd N, however, 
a qualitatively distinct feature is observed in the valley- 
Hall edge modes that we discuss at length in the follow- 
ing. Due to the requirement of absence of intervalley 
scattering, quantum valley Hall state is considered as a 
"weak" topological state, compared to the topologically 
protected quantum-Hall effect. This scenario resembles 
the requirement of time-reversal symmetry protection in 
Zi topological insulator. 

Figure QJa) plots the band structure of a zigzag-edged 
trilayer graphene ribbon in the presence of an interlayer 
potential difference U = O.lt. One can clearly observe the 




FIG. 1: (Colour online) (a) Band structure of a zigzag- 
terminated trilayer graphene ribbon in presence of an inter- 
layer potential difference proportional to U = O.lt. Letters 
'A-F' label the six edge states inside the bulk gap with the 
same Fermi energy, (b) Edge modes of the edge states la- 
belled in (a). Spins are doubly degenerate and valleys are 
associated with the edge states. Note that there are two pairs 
of valley-helical edge states at the upper boundary, while only 
one pair is localized at the lower boundary. 



valley-Hall edge states in the vicinity of each valley in- 
side the bulk gap. At charge neutrality, the edge modes 
in the system are formed by the left- and right-going 
states within the same valley (A, B and E, F) located at 
opposite edges as illustrated schematically in Fig. QJb). 
As soon as the Fermi level is shifted from neutrality, 
the system acquires two additional edge modes, labelled 
for electron-doped case with letters C, D in Fig. [Ha). 
This additional pair of edge modes forms two counter- 
propagating channels with opposite valley flavours, and 
is located at the same boundary of the ribbon, giving rise 
to a net valley-polarized current. The real-space edge lo- 
cation can be reversed by changing the direction of the 
external electric field. At the same time edge asymme- 
try of electron-hole wave functions allows for the control 
of the spatial distribution of edge modes through carrier 
doping. 

Here, we give an intuitive picture on how the valley- 
Hall edge modes emerge in trilayer graphene. In the ab- 
sence of interlayer coupling, a gated trilayer graphene is 
composed of three single layer graphene stacks with their 
electronic bands relatively shifted by the external poten- 
tial difference. When the interlayer coupling is further 
included, gaps open at the bulk band crossing points. 
Simultaneously, a pair of edge mode is formed at each 
valley point, i.e., bands labeled by A,B at valley K and 
E,F at valley K'. Based on the fact that A,F (B,E) are 
located at the same boundary, we can attribute that the 
band labeling with A,F(B,E) originates from one of the 
top (bottom) flat band. And the central flat bands con- 
nect with the bulk conduction (valence) bands to form 
the special band labeled with C,D, which gives rise to 
the unbalance edge mode. 

Therefore, we find an interesting scenario where both 
the valley polarized current directions as well as their 
location at a given edge can be appropriately switched 
through electric gating or controlling the carrier density. 




FIG. 2: (Colour online) Band structures of zigzag [(a) and 
(c)] and armchair [(b) and (d)] trilayer graphene ribbons at 
a fixed interlayer potential difference U — O.li with differ- 
ent Rashba SOC. (a)-(b): tn = 0.05t, six edge states, shown 
in blue (thick) lines, appear at each valley in zigzag ribbon; 
no gapless edge state exists inside the bulk gap of armchair 
ribbon, though there are some emerging gapped edge bands. 
(c)-(d): ta = 0.12£, one more pair of edge states are induced 
at each valley of zigzag ribbon, and gapless edge states are 
now formed inside the bulk gap of armchair ribbon. 



The above anomalous features of the unbalanced edge 
modes in ABC trilayer graphene are preserved when the 
system is brought into a topological insulator phase. 

Zi topological insulator edge modes — Spin-orbit cou- 
pling strengths for carbon atoms in graphene were esti- 
mated to have an extremely small value of the order of 
1CP 7 eV, rendering an almost negligible effect for both 
intrinsic and Rashba SOC in graphene under realistic 
condition a 16 ' 17 . However, interactions with substrates 18 
or adatomsi^ that introduce an additional inter-atomic 
effective electric field can increase the Rashba-type SOC 
to energy scale orders of meV. Interplay of Rashba SOC 
with layer inversion symmetry breaking potential in bi- 
layer graphene was shown to trigger an interesting phase 
transition from a quantum valley- Hall phase into a valley- 
protected topological insulator phased. We show that 
a similar topological phase transition is also found in 
gated trilayer graphene, but with additional novel fea- 
tures. The effect of a Rashba SOC is to split the spin 
degeneracy of the bands and lead to an eventual forma- 
tion of time-reversal invariance protected gapless edge 
modes for sufficiently strong Rashba SOC tR. In Fig. [3J 
we show the band structure evolution as a function of tn 
for both zigzag [panels (a) and (c)] and armchair [panels 
(b) and (d)] trilayer ribbons at a fixed potential differ- 
ence U = O.lt. Edge states are plotted in blue (thick) 
lines to distinguish from the bulk bands in black (thin) 
lines. 
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FIG. 3: (Colour online) Schematic plot of the edge modes 
corresponding to the conventional QVHI phase and strong 
TI phases. (a) Four (two) pairs of valley-helical edge 
states are localized at the upper (lower) boundary. When 
they encounter the armchair edge the inter-valley scattering 
backscatters the edge state associated with valley K to the 
counter-propagating edge state encoded with valley K' . (b) 
The major difference from panel (a) is that one more pair 
of time-reversal invariance protected spin-helical edge states 
emerges at each boundary of the zigzag or armchair trilayer 
graphene ribbon. Note that in the zigzag ribbon geometry, 
all the edge modes are associated with both spin and valley 
degrees of freedom. 



When a small Rashba SOC is introduced, i.e., £r = 
0.05£, the bulk band gap of the system starts to decrease 
and the spin degeneracy of the bulk bands is lifted. In 
Fig-I21a) for the zigzag ribbon, one can observe that three 
spin-degenerate pairs of edge states are split into six pairs 
of non-degenerate edge modes, still preserving the uneven 
spatial distribution at opposite edges: four pairs of edge 
states propagating along one edge, while only two pairs 
travelling along the other edge, as shown in Fig. [3{a). 
Simultaneously, edge modes start to emerge within the 
bulk gap for the armchair ribbon as shown in Fig. [2jb). 

With further increase of Rashba SOC, the bulk band 
gap continues to decrease while the system remains in 
the quantum valley-Hall insulator (QVHI) phase. The 
system finally reaches a critical point at £r = 0.094i, 
where the bulk gap completely closes. Beyond this point, 
the bulk band gap reopens, which suggests a topologi- 
cal phase transition. In the following, we demonstrate 
the novel characteristics of the resulting edge modes 
in zigzag-terminated trilayer graphene ribbon after the 
phase transition. 

Figures Hfc) and HJd) show the band structures for a 
larger Rashba SOC tR = 0.12t. In zigzag-terminated tri- 
layer graphene, we find a different behaviour with respect 
to the bilayer case, where one pair of edge states merge 
and disappear into the bulk bands^S. Here, a new pair 
of edge states emerges from the bulk at both valleys K 
and K', giving rise to a total of eight pairs of edge modes 
inside the bulk gap [see Fig. Hfc)]. The resulting edge 
modes have a rather surprising spatial arrangement as we 
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FIG. 4: (Colour online) (a) Phase-diagram of ABC trilayer 
graphene in the parameter space of tn and U. Colors repre- 
sent the bulk gap size. Two phases are clearly separated: con- 
ventional QVHI phase with Z2 = 0, and the two-dimensional 
(2D) TI phase with Z2 = 1. The dashed line trace corre- 
sponds to the potential difference U = O.lt, which is exhib- 
ited in panel (b). The bulk gap shown in panel (b) closes 
and reopens once in the chosen scale, signaling a topologi- 
cal phase transition from QVHI phase to 2D TI phase. Inset 
shows that the gap closing point is not at the exact K point. 
The solid (blue) curves are the bulk band structure of gated 
trilayer graphene around valley K at the critical Rashba SOC 
t c R — 0.094t; while the dashed lines plot the band structure 
of single layer pristine graphene, where the crossing point is 
exactly at the K point. 



show in the schematic plot in Fig.^b): five pairs of edge 
states are located at the upper boundary, while three 
pairs are located at the lower boundary. The odd pairs 
of spin-helical edge states propagating in a time-reversal 
invariant system indicate a topological insulator state. In 
this way, we provide an intuitional picture showing that 
unequal numbers of edge state pairs at opposite sample 
boundaries can exist in a graphene-based TI system due 
to the protection of large valley separation. 

A further confirmation of the topological insulator is 
the appearance of two pairs of gapless edge states in the 
armchair-terminated trilayer graphene ribbon: there is 
one (odd) pair of edge states flowing along each bound- 



ary of the armchair ribbon [see the vertical direction of 
Fig. HJd)]. Based on the above analysis, we can obtain 
a schematic diagram of the topological insulator edge 
states in trilayer graphene as illustrated in Fig. \T[b): 
there is one pair of time-reversal invariance protected 
edge states circulating along any boundaries, while the 
remaining edge modes can only propagate along zigzag 
boundaries due to the inter-valley scattering present in 
an armchair edge. Note that the five (three) pairs of 
edge modes at the upper (lower) boundary are equally- 
weighted and nontrivial. 

Another important signature of TI is the Z-i topolog- 
ical number that characterizes the band topolog y 12 ' 21 . 
Using the method described in Ref. [22(, we numerically 
compute the Z2 topological number for our system, and 
the results show that Z-i = before the phase transition, 
while Z% = 1 after the phase transition, consistent with 
our band structure analysis. 

Phase-diagram — To give a complete understanding on 
how the QVHI phase evolves as functions of ta and U, 
we present a "phase-diagram" of the bulk band gap A in 
Fig-BJa). Colour maps are used to indicate the bulk gap 
magnitude. The two separate regimes correspond to a 
conventional QVHI phase with Z-i = and cr" = 3e 2 //i 
(considering both spins) and a topological insulator phase 
with Z2 = 1, respectively. As a guide to the eye, we plot 
the bulk gap dependence as a function of t^ at a fixed 
potential difference U = O.li in Fig. QJb) that shows one 
gap closure and reopening as we discussed before. 

Different from the topological phase transition in bi- 
layer graphene occurring exactly at K and K' points, 
in trilayer graphene the bulk band gaps close at some 
points away from the exact valley K/K' points [see the 
Inset of Fig. H(b)]. As a consequence, the low-energy 
continuum model can not correctly capture this nontriv- 
ial topological insulator phase. Therefore, although the 
obtained topological insulator phase in trilayer graphene 
is still associated with valley degrees of freedom, one can 
not calculate a well-defined valley-Hall conductivity from 
the corresponding continuum model. 

Summary and discussions — We have studied the spa- 
tial imbalance and valley current polarization of the edge 
modes in gated trilayer graphene. The spatially uneven 
distribution of edge modes arises from the half-integer 
quantum valley-Hall conductivity of trilayer graphene 
that challenges the conventional understanding of how 
edge modes are related with the bulk topology. These 
features can in principle be explored in a doubly gated 



trilayer device that would allow for a direct control of 
the edge modes, either by modifying the carrier dop- 
ing or reversing the sign of the interlayer potential dif- 
ference. Although perfect zigzag trilayer graphene may 
not be experimentally accessible in current conditions, 
it is argued that a substantial contribution of quantum 
transport in realistic samples of bulk gapped multilay- 
ers might still be mediated by valley-Hall edge modes 2 ^. 
In such cases, the experimental signatures of edge mode 
imbalance in trilayer graphene discussed in this Rapid 
Communication should have measurable consequences in 
electron transport experiments, e.g. in the form of orbital 
moments^, generated by the imbalanced current carrying 
edge modes. 

Another noteworthy finding reported in the present 
work is that the trilayer graphene can be turned into a 
topological insulator phase in the presence of sufficiently 
large Rashba SOC, where the distribution of the edge 
modes at opposite boundaries remains uneven. Due to 
the special structure of zigzag ribbon without intervalley 
scattering, we show that the numbers of edge mode pairs 
at both boundaries are odd, and most importantly, they 
are unequal, i.e. five pairs at one boundary while three 
pairs at the other. When these edge states encounter the 
armchair edge, only one pair of topologically protected 
edge modes can survive due to the presence of strong 
intcr-valley scattering. 

Finally, we discuss the stability of the unbalanced edge 
states in the presence of external disorders. As demon- 
strated in Ref.—, the valley-Hall edge modes can be eas- 
ily destroyed by short range disorders due to the back 
scattering between valleys. On the contrary, it is found 
that long range (smooth) disorders can strongly suppress 
the back scattering. Moreover, in graphene the impurity 
scattering in graphene mainly arises from the long-range 
Coulomb scatterers. Therefore, the proposed unbalanced 
edge modes are robust against the smooth disorders and 
should be detectable in a realistic zigzag trilayer graphene 
ribbon. Recently, it is reporte d 25 ' 26 that surface states in 
weak topological insulators having even number of Dirac 
cones are very robust as long as the perturbation does 
not break the time reversal symmetry. 
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